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I.  INTRODUCTION 


The  designers  of  aircraft  are  rapidly  adopting  glass  cockpit 
technology  where  conventional  electromechanical  and  pneumatic  in¬ 
struments  are  being  replaced  by  cathode  ray  tubes  (CRTs)  for  pre¬ 
sentation  of  information  to  the  pilot  and  other  crew  members. 
Further,  head-up  displays  (HUD)  are  being  adopted  as  the  primary 
flight  reference  for  instrument  meteorological  conditions.  This 
technology  influx  has  created  the  potential  for  new  and  unique 
formats  by  which  information  critical  to  flight  and  mission  suc¬ 
cess  is  conveyed  to  the  flight  crew.  In  fact,  single  seat, 
night/all-weather  low  altitude  missions  are  being  flown  success¬ 
fully  only  because  of  this  technology.  The  steering  group  for 
night  attack,  as  an  advisory  group  for  the  introduction  of  LAN- 
TIRN  (Low  Altitude  Navigation  and  Targeting  Infrared  System  for 
Night),  prioritized  the  head-up  display  as  a  critical  technologi¬ 
cal  element  for  this  mission  (J.)  .  In  the  LANTIRN  mission,  the 
flight  is  conducted  essentially  with  sole  reference  to  the  cock¬ 
pit  display  environment  and  the  HUD  is  a  critical  component.  Con¬ 
sequently,  the  influence  of  the  HUD  on  flight  information  proces¬ 
sing  and  manual  flight  control  is  critical. 

The  HUD  is  an  outgrowth  of  the  reflective  gunsight  of  World 
War  II.  In  such  gunsights,  the  aiming  symbol  was  generated  as  a 
collimated  beam  of  light,  projected  upwards,  and  reflected  toward 
the  pilot  by  a  semi-transparent  mirror  placed  in  his  field-of- 
view  (FOV)  through  the  windshield.  If  the  design  is  correct,  the 
pilot  will  see  the  symbols  floating  in  his  view  of  the  outside 
scene.  The  image  of  the  symbols  can  be  focused  to  form  a  virtual 
image  which  appears  to  lie  in  the  same  plane  as  the  outside 
visual  scene.  From  lead-computing  gunsights,  the  next  step  was 
to  place  flight  information  in  the  virtual  image. 

The  reasons  for  providing  a  head-up  display  are  seemingly 

intuitive: 


•  A  head-up  display  can  reduce  pilot  workload  when 
the  piloting  task  requires  head-up,  outside-the- 
cockpit  flight  references. 

•  Improvements  in  accuracy  and  efficiency  occur  from 
the  overlay  of  HUD-presented  data  with  the  exter¬ 
nal  visual  scene. 

Much  of  the  early  development  of  HUDs  took  place  at  the  UK's 
Roya  i  Aircraft  Establishment  (RAE)  in  the  late  1950s  and  early 
1960s.  These  early  studies  indicated  that  a  HUD  need  not  be  con- 
forma!  to  the  real  world,  but  rather  only  an  approximate  overly¬ 
ing  of  symbols  and  real  world  cues  was  required(2).  Part  of  this 
may  have  been  the  result  of  a  lack  of  technology  to  reliably  gen¬ 
erate  a  conformal  contact  analog. 


The  early  work  at  the  RAE  was  based  on  extensive  flight  test 
and  simulator  experiments.  Most  of  the  conclusions  were  based  on 
a  performance  metric,  that  is,  the  success  criteria  for  a  display 
was  based  on  the  minimum  tracking  error.  The  ability  of  the  pi¬ 
lot  to  monitor  the  display  and  his  own  performance  was  not  usual¬ 
ly  considered.  Naish,  in  one  simulator  study,  did  purposely  mis¬ 
guide  the  subject  pilots  to  a  touchdown  off  the  runwayO)*  He 
found  that  the  subject  pilots  tended  to  ignore  the  HUD  and  fly  by 
the  real  world  cues  as  they  became  available.  Similar  experi¬ 
ments  were  carried  out  by  NASA  in  the  1970s  with  similar  conclu¬ 
sions  . 

In  two  flight  experiments,  using  the  mid-air  retrieval  sys¬ 
tems  (MARS),  pilots  tended  to  believe  the  HUD  cues  and  ignored 
the  real  world  cues  H,5J  .  This  points  out  one  difficulty  with 
the  use  of  simulators  to  predict  flight  behavior.  Pilots  may 
well  use  different  strategies  when  flying  a  simulated  mission 
than  they  would  in  flight. 

In  the  1980s,  the  McDonnel 1 -Douglas  MD-80  airplane  used  a 
HUD  as  a  monitor  for  the  Category  II  autoland  system.  The  HUD 
was  intended  to  be  used  as  a  monitor,  but  under  some  circumstanc¬ 
es  the  HUD  wouid  show  a  perspective  runway  scene  to  be  used  as  a 
control  display.  Since  the  MD-80  does  not  have  an  inertial  navi¬ 
gation  system  installed',  some  errors  were  found  in  the  runway 
scene.  Pilots  reported  that  the  display,  when  mis-aligned  caused 
a  strong  tendency  to  steer  off  of  the  runway  during  the  landing 
roll.  As  a  result,  the  runway  display  was  changed  to  a  much  less 
compelling  version(6J. 

During  the  1970s,  the  USAF  installed  a  modified  Sundstrand 
HUD  in  a  Northrop  T-38.  This  HUD,  called  the  light  line,  showed 
a  virtual  image  of  the  airplane's  flight  path  as  an  apparent  beam 
of  light.  Unfortunately,  the  accuracy  of  the  T-38  heading  gyros 
was  such  that  the  lightline  HUD  did  not  line  up  accurately  enough 
with  the  airplane's  actual  flight  path  and  the  project  was  termi¬ 
nated  (7  )  . 

These  early  results  all  lead  to  some  fundamental  concern 
about  the  use  of  HUDs  as  flight  displays.  One  question  is  clear¬ 
ly,  how  accurate  does  the  airplane's  gyro  platform  have  to  be? 
Many  military  HUD-equipped  airplanes  have  inertial  navigation 
systems  which  appear  to  be  satisfactory.  Other  candidate  air¬ 
planes  for  HUDs  --  transports,  trainers,  etc.  --  do  not  have  in¬ 
ertial  navigation  capability.  For  these  airplanes,  we  need  to 
determine  the  minimum  accuracy  required  for  the  gyro  platform. 

From  the  MD-80  history,  it  appears  that  the  HUD  symbology 
may  affect  the  answer  to  this  question.  A  very  realistic  contact 
analog  display  was  unacceptable  with  the  heading  errors  in  the 
MD-80,  while  a  less  realistic  symbolic  runway  symbol  (the  side¬ 
lines  of  a  very  short,  wide  runway)  was  satisfactory. 


<d  t) 


The  present  study  was  designed  to  address  the  issue  of  how 
accurately  does  the  runway  symbol  have  to  be  placed  to  produce  an 
cceptable  HUD.  To  this  end,  an  in-flight  experiment  was  con- 
ucted  using  the  USAF  variable  stability  NT-33A  aircraft.  The 
NT- 3  3 A  aircraft  is  equipped  with  a  programmable  head-up  display 
which  is  part  of  the  Display  Evaluation  Flight  Test  (DEFT)  sys¬ 
tem.  Two  experiments  were  conducted.  One  was  to  investigate  the 
dynamic  response  requirements  of  the  HUD  flight  symbols  and  the 
second  to  determine  the  accuracy  requirements  for  instrument  ap¬ 
proaches.  The  two  tasks  were  conducted  simultaneously.  The  dy¬ 
namic  response  study  has  been  reported  el sewhere  (jl,  9J  .  The  accu¬ 
racy  study  is  the  subject  of  this  report. 


II.  OBJECTIVES 


The  global  objective  of  this  study  was  to  investigate  accu¬ 
racy  needs  of  the  pilot  when  viewing  real  world  cues  and  HUD  cues 
simultaneously.  The  head-up  display  by  its  nature  and  format  is 
a  compelling  instrument.  If  the  HUD  symbol  and  its  real  world 
counterpart  differ,  this  must  not  create  problems  either  from  the 
point  of  view  of  the  pilot  ignoring  the  HUD  because  of  loss  of 
confidence  or,  more  importantly,  ignoring  the  real  world  cues  be¬ 
cause  of  over-confidence  in  the  HUD  or  because  the  HUD  creates  an 
illusion  that  is  too  compelling. 

This  task  examined  various  contact  analog  runway  displays 
for  the  instrument  apprjach  and  landing  task  and  also  the  poten¬ 
tial  hindrance  caused  by  the  display  in  the  transition  to  visual 
flight  reference.  The  evaluation  included  intentional  inaccura¬ 
cies  in  the  projection  of  the  HUD  displayed  runway  symbology  as 
it  appeared  to  overlay  the  actual  runway.  In  this  manner,  the 
pilot's  ability  to  transition  from  the  HUD  to  outside- the-cock- 
pit,  visual  flight  reference  will  be  tested  under  varying  degrees 
of  accuracy  for  which  sensors  can  determine  an  aircraft's  posi¬ 
tion  and  attitude. 

The  landing  task  was  chosen  because  it  is  considered  to  be  a 
significant  issue  and  because  it  was  felt  that  weapon  delivery 
accuracy  would  be  driven  by  the  weapon  accuracy  specification. 

The  objectives  of  the  subject  task  (symbol  accuracy  require¬ 
ments)  were: 

•  Investigate  pilot  performance  and  judgment  during 
the  IMC/VMC  transition  from  an  instrument  landing 
system  approach  under  conditions  of  HUD  symbol 
inaccuracies . 

•  Determine  if  there  is  any  tendency  for  pilots  to 
follow  the  HUD  and  ignore  real  world  cues  during 
the  IMC/VMC  transition  or  thereafter. 

•  Generate  pilot  opinion  ratings  and  comment  data 
regarding  the  optimum  head-up  display  for  the  in¬ 
strument  approach  and  landing  task. 

•  Examine  different  runway  symbol  types  and  their 
effect  on  the  instrument-to-visual  transition  for 
landing  in  the  face  of  sensor  limitations  and  er¬ 
rors  and,  also,  in  their  ability  for  providing 
approach  guidance  information. 


III.  EXPERIMENT  DESIGN 


An  in-flight  investigation  of  Head-up  Display  (HUD)  symbol 
accuracy  requirements  was  conducted  using  the  USAF  Flight  Dynam¬ 
ics  Laboratory  variable  stability  NT-33A  aircraft.  The  NT-33A 
aircraft  is  an  inflight  simulator  equipped  with  a  programmable 
head-up-display  which  is  part  of  the  DEFT  system.  The  handling 
qualities  and  display  characteris t ics  of  the  inflight  simulator 
can  be  easily  altered  for  experimentation  purposes. 

The  NT-33  variable  stability  system  capabilities  are  used  to 
simulate  different  aircraft  handling  qualities  characteristics 
for  the  evaluation  pilot  who  sits  in  the  front  seat  of  the  two- 
seat  NT-33.  The  mechanization  of  the  NT-33  simulation  is  not  un¬ 
like  a  ground-based  simulation;  however,  the  NT-33  simulation  is 
essentially  unconstrained  in  motion  response  and  the  visual  envi¬ 
ronment  is  real.  The  motion  of  these  simulated  configurations 
are  sensed  by  appropriate  transducers.  These  signals  are  condi¬ 
tioned  and  processed  in  the  programmable  display  system  of  the 
NT-33  for  head-up  display  presentation.  Instrument  meteorologi¬ 
cal  conditions  (IMC)  are  effectively  and  safely  simulated  by  us¬ 
ing  a  blue/amber  system.  The  front  canopy  of  the  NT-33A  is  cov¬ 
ered  with  an  amber  plastic  sheet;  when  the  front  seat  pilot  low¬ 
ers  his  blue  visor,  the  complimentary  colors  produce  an  almost 
completely  black  outside  environment.  Flight,  in  this  instance, 
is  conducted  by  the  evaluation  pilot  solely  from  the  cockpit  dis¬ 
play  environment.  The  simulated  aircraft  configurations,  HUD 
format,  symbols,  dynamic  response  characteristics,  and  evaluation 
task  were  selected  are  tailored  to  satisfy  the  objectives  of  this 
experiment . 


Experiment  Variables 


This  experiment  consisted  of  two  tasks:  Task  A  was  an  inves¬ 
tigation  of  the  effect  of  head-up  display  symbol  dynamic  response 
characteristics  on  flying  qualities;  Task  B  was  an  investigation 
of  symbol  accuracy  requirements  as  they  pertain  to  the  accuracy 
by  which  a  contact  analog  runway  display  symbol  overlays  the  real 
world  and  the  potential  hindrance  introduce  by  the  HUD  in  the  pi¬ 
lot's  transition  from  instrument- to-visual  flight  reference. 
While  the  two  experiments  were  conducted  one  the  same  flight,  in 
reality  they  were  two  separate  experiments. 


The  experiment  variables  for  this  study  were: 


•  (Intentional)  HUD  symbology  errors, 


Runway  display  symbol. 
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With  the  experiment  objectives  and  variables  thus  defined,  the 
experiment  configurations  and  mechanization  are  documented  in  the 
following  sections. 


B .  USAF/FDL  NT-33A  Research  Aircraft 

The  NT- 3 3 A  aircraft  is  an  extensively  modified  Lockheed  T-33 
jet  trainer.  It  is  owned  by  the  Flight  Dynamics  Laboratory  and 
operated  under  contract  by  Calspan  Corporation.  The  front  seat 
control  system  of  the  NT-33A  has  been  replaced  by  a  full 
authority  fly-by-wire  flight  control  system  and  a  variable 
response  artificial  feel  system.  The  evaluation  pilot,  who  sits 
in  the  front,  cockpit,  controls  the  aircraft  through  a  standard 
centorstdck  and  rudder  pedal  arrangement  or  a  sidestick  control¬ 
ler  installed  on  a  right-hand  console  as  an  alternate  pitch  and 
roll  controller. 

The  front  seat,  fly-by-wire  control  system  and  variable  re¬ 
sponse  feel  system  can  be  programmed  to  simulate  several  aircraft 
configurations.  The  system  operator  in  the  rear  cockpit,  who  al¬ 
so  acts  as  safety  pilot,  controls  the  HUD  and  aircraft  configu¬ 
ration.  It  is  important  to  note  that  the  evaluation  pilot  cannot 
feel  the  control  surface  motions  due  to  the  actions  of  the  vari¬ 
able  stability  system  signals  in  the  NT-33.  During  this  experi¬ 
ment,  the  evaluation  pilot  had  no  prior  knowledge  of  the  configu¬ 
ration  characteristics. 

Safety  features  are  an  essential  and  integral  part  of  the 
NT-33  research  aircraft.  Continuous  safety  monitors  activate  an 
automatic  safety  trip  system  to  disconnect  the  evaluation  pilot 
from  the  fly-by-mire  control  system  before  unsafe  flight  condi¬ 
tions  or  aircraft  attitudes  occur.  Aircraft  control  reverts  to 
the  safety  pilot  who  occupies  the  rear  seat  with  unmodified  T-33 
flight  controls.  The  safety  pilot,  who  also  acts  as  the  systems 
operator,  provides  an  additional,  redundant  margin  of  safety  by 
his  ability  to  disengage  the  variable  stability  system  manually. 


Details  of  the  simulation  mechanization  are  provided  in  Ref¬ 
erences  (8  and  10) . 


C .  Head-Up  Display  (DEFT) 

A  fully  programmable  head-up  display  (HUD)  which  is  part  of 
the  display  evaluation  flight  test  (DEFT)  system  is  installed  in 
the  front  cockpit.  The  DEFT  system  is  described  el sewhere (11). 
Eight  distinct  display  programs  are  available  for  loading  in 
flight.  Within  each  program  are  data  stored  for  six  runways  for 
landing  and  approach  evaluations. 

The  HUD  optics,  f ield-of-view,  and  design  eye  reference 
point  (DERP)  were  not  experimentally  varied  nor  formally  evaluat¬ 
ed.  These  hardware  items  were  the  nominal  DEFT  system  and  are 
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unique  to  the  HUD  and  its  installation  In  the  T-33.  Unfortunate¬ 
ly,  the  HUD  FOV  and  DERP  are  not  ootimal.  The  instantaneou s  FOV 
is  limited  to  approximately  16  deg  in  azimuth  and  18  deg  verti¬ 
cal.  The  DERP  is  low  and  hence,  a  potential  hindrance  to  the 
pilot.  The  DERP  .is  restricted  because  of  ejection  envelope  and 
panel  mounting  constraints  in  the  T-33  installation.  These  defi¬ 
ciencies  were  noted  by  all  of  the  evaluation  pilots.  DERP  loca- 
r  .on  and  limited  FOV  problems  are  typical  of  many  HUD-equipped 
a  i  re  ra  f  t. . 

instrument  meteorological  conditions  were  simulated  using  a 
blue/amber  system.  An  amber  vinyl  plastic  sheet  covered  the  in¬ 
side  front  half  of  the  NT-33  canopy.  Blue  snap-on  visors  were 
provided  to  the  evaluation  pilots.  The  compl ementa ry  colors, 
with  the  blue  visor  lowered,  effectively  present  instrument  me¬ 
teorological  conditions  to  the  evaluation  pilot,  yet  do  not  over- 
iy  restrict  the  visual  conditions  of  the  safety  pilot. 


The  blue-amber' 
tried  systems: 


system  has  several  advantages  over  previously 


It  is,  perhaps,  the  only  means  of  simulating  IMC 
while  using  a  HUD  which  will  retain  adequate  vis¬ 
ibility  for  the  safety  pilot. 

The  evaluation  pilot  cannot  cheat  the  visual  re¬ 
striction  (such  as  is  possible  IMC  visors). 

The  blue/amber  is  less  cumbersome  than  canopy 
drapes . 

The  TMC  restriction  can  be  quickly  and  easily  re¬ 
moved  . 


As  in  any  simulation,  the  blue/amber  simulation  of  instru- 
icnc  conditions  is  not  without  its  limitations.  These  imperfec- 
ions  are  itemized  for  proper  interpretation  of  the  experiment 


suit s : 


The  ability  to  transition  quickly  from  IMC  to  VMC 
is  a  disadvantage  as  well  as  advantage  of  the 
biuo/amber  simulation.  Night  flying  instantly 
can  become  full  daylight  VMC.  This  situation  is 
no';  completely  realistic  and  some  visual  accommo¬ 
dation  is  needed  after  the  transition.  The  HUD 
intensity,  usually  set  for  night  conditions, 
needs  to  be  increased  after  the  transition  to  VMC 
to  be  legible. 


The  blue/ambor  technique  does  not  provide  any 
shade:;  of  gray  between  full  VMC  and  IMC.  Also, 
the  majority  of  flights  were  flown  in  very  good 
weather  conditions  (always  under  visual  flight 
rules)  and  typically  good  visibility  (greater 
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than  10  miles).  During  a  landing  approach  evalu¬ 
ation,  a  somewhat  unrealistic  situation  occurs 
because  the  breakout  from  instrument  conditions 
in  the  blue/amber  simulation  yields  limitless 
visibility.  Approach,  threshold,  and  runway  edge 
marker  lights  are  absent  in  this  scenario. 


Although  the  IMC  simulation  was  not  perfect,  it  did  provide 
a  constant  and  consistent  IMC  simulation  for  which  to  test  head- 
up  display  systems. 


Aircraft  Conf iaurat ions  Flown 


Two  flight  phases  were  flown  on  each  sortie; 


up-and-away  (symbol  response  study) 


power  approach  and  landing (symbol  re¬ 
sponse  study  and  display  accuracy  study, 
this  study) 


Details  of  the  evaluation  task,  procedures,  and  flight  phase  are 
given  in  Section  IV. 


In  the  power  approach  flight  phase,  two  fighter  and  one  trans¬ 
port-type  aircraft  configuration  were  simulated.  The  fighters 
were  identified  as  Configurations  D  and  E,  and  the  transport  as 
Configuration  T.  For  the  purposes  of  this  study,  the  differences 
were  slight.  All  three  were  designed  to  be  good,  level  1  by  MIL- 
F-8785C,  (1_2)  aircraft.  Configuration  E  would  be  termed  slightly 
sluggish.  Details  of  the  aircraft  and  control  dynamics  for  each 
configuration  can  be  found  in  el sewhere ( 8 , 9 ) . 


Display  Configurations 


For  this  study,  intentional  misalignments  between  the  real 
runway  and  a  contact  analog  HUD  runway  symbol  were  introduced. 
Three  different  runway  symbols  were  investigated. 


A  generic  head-up  display  format  was  used  as  the  baseline 
display  format.  This  generic  HUD  format  was  used  to  keep  display 
clutter  to  a  minimum.  Mission  specific  information  was  not  pro¬ 
grammed  nor  was  it  felt  to  be  required.  A  generic  HUD  was  suit¬ 
able  for  our  purposes  since  other  tasks  under  this  investigation 
of  HUD  requirements  were  intended  to  investigate  optimal  display 
formats  and  presentations.  The  generic  HUD  format  used  as  the 
baseline  display  is  sketched  in  Figure  1. 


The  primary  features  of  the  display  are  the  digital  air¬ 
speed,  altitude,  and  heading  information  readout,  with  a  1-to-l 
pitch  ladder.  The  pitch  ladder  is  marked  in  5  degree  increments; 
the  pitch  ladder  below  the  horizon  is  dashed  whereas  a  solid  line 


is  usod  for  positive  pitch  attitudes.  The  ladder  tails  point  to 
the  horizon.  Negative  signs  are  not  shown.  This  format  is  es¬ 
sentially  the  nominal  DEFT  system,  which  is  similar  in  many  re¬ 
spects.  to  the  presentation  used  in  F-18  aircraft. 


The  waterline  marker  is  a  fixed  reference  approximately  par¬ 
allel  to  the  aircraft  waterline  reference.  The  nominal  velocity 
vector  (FPM  for  flight  path  marker)  was  calculated  using  air  mass 
quantities  (i.e.,  FPM  -  THETA  -  ALPHA).  When  the  velocity  vector 
was  freed  in  azimuth,  it  displayed  sideslip.  For  the  majority  of 
the  program,  a  declutter  option  was  available  to  the  pilot.  He 
had  the  option  through  a  pushbutton  on  the  front  seat  instrument 
panel  to  select  either  of  three  presentations  of  the  waterline 
and  velocity  vector: 


Decl utter  ( 0 ) : 


waterline  not  displayed; 
velocity  vector  displayed 
(caged  in  azimuth) 
waterline  displayed; 
velocity  vector  displayed 
(free  in  azimuth) 
waterline  displayed; 
velocity  vector  not  displayed. 


Only'  aii'  mass  velocity  vectors  were  used  in  this  experiment. 
This  was  done  for  two  reasons.  First,  since  we  would  be  intro¬ 
ducing  deliberate  errors  into  the  runway,  using  an  inertial  velo¬ 
city  vector  would  show  the  error  to  the  pilot  immediately.  If  an 
air  mass  velocity  vector  were  used,  the  pilot  could  not  tell  from 
the  HUD  whether  there  was  an  error  in  the  runway  or  if  there  was 
a  strong  wind. 


The  second  reason  was  to  avoid  problems  because  of  the  rela- 
ively  slow  (10  Hz)  update  rate  of  the  inertial  velocity  vector. 


Several  display  options  were  evaluated  briefly.  These  in- 

c ud^d : 


Potential  Flight  Path  Marker 


Angle  of  Attack  Bracket 


The  potential  flight,  path  marker  is  a  display  feature  adopt- 
f  rom  the  unique  all-analog  Klopfstein  display  (1_3)  .  The  po- 
it;ui  flight,  path  marker  is,  in  essence,  the  rate  of  change  of 
■>  flight  path.  It  is  indicated  by  triangular  carets  at  the 
i phe ry  of  the  display  (Figure  2).  The  potential  flight  path 
-kora  indicate  where,  if  the  controls  were  left  unchanged,  the 
g  n  t  path  will  eventually  move.  The  potential  flight  path 
'kors  can  be  thought  of  as  thrust/throttle  flight  directors, 
vi  the  potential  flight  path  and  flight  path  markers  are 
gned,  t hru s t  equals  drag  and  the  aircraft  is  stabilized  on  the 
i  i  ca  t  ed  f  1  i  gh  t.  pa  t.  h  angle. 
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The  angle  of  attack  bracket  display  provides  an  explicit 
target  angle  of  attack  marker  for  the  power  approach  (Figure  3). 
The  sense  of  the  bracket  is  fly-to.  The  bracket  shows  a  devia¬ 
tion  of  one  degree  from  the  target  angle  of  attack.  If  the  angle 
of  attack  exceeds  the  target  by  greater  than  1.2  deg,  a  digital 
readout  of  the  angle  of  attack  is  presented  in  the  lower  left 
hand  corner  of  the  display.  The  bracket  is  drawn  with  respect  to 
the  waterline  marker. 


>ol  Accuracy  and  Display  Variables 


It  was  necessary  to  implement  intentional  errors  in  the  cal¬ 
culation  and  hence,  display  of  a  contact  analog-type  approach  and 
landing  symbology. 

A  generic  contact  analog  runway  overlay  was  programmed  as 
the  nominal  case.  This  display  was  created  by  augmenting  the 
generic  HUD  format  used  in  the  up-and-away  flight  condition  with 
landing  guidance  information.  For  the  landing  and  approach  eval¬ 
uation,  a  mode  logic  was  implemented.  Once  the  approach  mode  was 
initiated,  the  automatic  mode  sequencing  of  DEFT  was  activated. 
This  sequence  was  started  upon  safety  pilot  activation  of  the  AP¬ 
PROACH  mode.  Initially,  raw  ILS  data  drive  deflection  needles 
displayed  on  the  HUD.  The  needles  are  split  (i.e.,  glideslope 
needle  separated  from  the  localizer).  Distance  to  the  glidepath 
intercept  point  with  the  runway  is  digitally  shown  in  the  lower 
right-hand  comer  of  the  display  as  a  DME  readout.  The  needles 
are  drawn  with  respect  to  the  velocity  vector.  Once  inside  the 
approach  cone,  the  needles  are  blanked  and  a  contact  analog  run¬ 
way  symbol  is  drawn.  Under  normal  conditions,  this  symbol  would 
overlay  the  real-world  runway  (Figure  4).  This  symbol  is  sized 
to  project  an  area  equal  to  the  real  runway  when  viewed  from  the 
airplane  cockpit.  The  size,  shape,  and  orientation  of  this  run¬ 
way  symbol  changes  as  the  airplane  maneuvers  for  landing  to  re¬ 
tain  the  conformal,  1:1  relationship  and  the  proper  runway  per¬ 
spective  . 


The  philosophy  behind  the  contact  analog,  runway  landing 
symbology  is  that  the  line-up  for  landing  and  transition  from  in¬ 
side-  the-cockpi t  instrument  to  outside-the-cockpit  visual  flight 
reference  is  optimized  and  simplified.  For  instance: 

•  The  perspective  provided  by  the  runway  display  is 
a  surrogate  to  the  visual  runway  landing  scene, 
lhe  pilot's  experience  and  skills  in  the  visual 
landing  task  are,  consequently,  transferred  to 
the  instrument  approach  with  the  contact  analog 
runway . 
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•  Flight  director  guidance  information  is  not  need¬ 
ed;  the  pilot,  in  essence,  provides  the  necessary 
compensation  through  his  internal  thought  and 
visual  perception  mechanisms  as  he  normally  would 
in  a  visual  landing.  The  instrument  landing  task 
changes  from  one  of  flying  flight  director  or  ILS 
needles  to  the  usual  task  of  visual  landing. 

•  The  pilot's  visual  transition  from  this  display 
to  the  outside  world  for  landing  is  dramatically 
improved.  The  pilot  is  presumably  focusing 
significant  attention  to  the  runway  symbol  pro¬ 
jection  for  the  HUD;  once  the  instrument  con¬ 
ditions  abate,  the  actual  runway  and  visual  cues 
will  appear  under  the  HUD  runway  display. 

®  The  pilot's  cross  check  from  head-down  to  head-up 
for  visual  breakout  is  virtually  eliminated.  Al¬ 
so,  the  pilot  can  focus  his  attention  for  break¬ 
out  indicators  (runway  lights)  in  the  vicinity  of 
the  displayed  runway. 

Clearly,  the  contact  analog  runway  provides  several  impor¬ 
tant  advantages.  The  contact,  analog  runway,  such  as  the  unique 
all -analog  Klopfstein  format,  has  been  successfully  demonstrated 
and  used.  In  this  program,  the  runway  of  the  Klopfstein  all-ana- 
log  format  has  been  superimposed  on  the  generic,  digital  format 
HUD. 

The  last  bullet  highlighting  the  advantages  of  a  contact 
analog  runway,  also  points  out  a  potential  deficiency  of  this 
concept  and  is,  thus,  the  subject  of  this  task.  Visual  fixation 
on  the  head-up  display  has  been  often  reported;  if  there  are  er¬ 
rors  in  the  framing  or  display  of  the  runway  symbol,  can  the  pi¬ 
lot  transition  from  the  HUD  to  detect  these  errors  in  a  timely 
fashion  and  successfully  land  the  aircraft? 

The  objective  of  this  task  were,  thus: 

•  Investigate  pilot  judgment  and  potential  hin¬ 
drance  during  the  transition  from  the  inside-the- 
cockpit  instrument  to  outside-the-cockpit  visual 
flight  reference  from  an  instrument  landing  ap¬ 
proach  using  a  contact  analog-type  HUD  format. 

•  Examine  this  transition  behavior  in  the  face  of 
errors  in  the  displayed  runway  projections. 

•  Examine  this  transition  behavior  and  manual  ap¬ 
proach  performance  using  different  runway  sym¬ 
bol  s . 

Three  runway  symbols  were  examined  which  vary  in  perspective 
detail  and  attractiveness.  The  three  symbols  were  chosen  to  span 


a  range  of  potential  transition  hindrance,  display  attractive¬ 
ness,  and  fullness  of  perspective: 

•  Runway  Symbol  A:  Full  runway  outline  with  cen¬ 

terline  projection  identical  to  the  Klopf stein 
runway  symbol  (Figure  4).  This  runway  symbol  is 
sized  to  be  equal  to  the  actual  runway  length  and 
width.  (This  runway  symbol  was  used  in  the  power 
approach  task  evaluation  in  Task  1  of  this  exper¬ 
iment)  . 

•  Runway  Symbol  B:  Sidel ines-only  display  (Figure 

5) .  This  runway  symbol  is  a  fixed  size  and  rep¬ 
resents  more  of  an  error  box  rather  than  a  con¬ 
tact  analog  runway.  It  was  felt  that  this  dis¬ 
play  presented  less  of  a  potential  visual  transi¬ 
tion  hindrance.  Rather  than  being  a  runway  out¬ 
line,  the  symbol  indicates  to  the  pilot  that  the 
runway  is  somewhere  in  the  general  subtended 
area.  This  symbol  is  sized  to  be  equal  to  a  2000 
ft  runway  500  ft  wide.  The  same  symbol  also 
flown  (Symbol  B1)  which  was  the  same  length  (2000 
ft),  but  only  250  ft  wide. 

•  Runway  Symbol  C:  Centerline  only  display  (Figure 

6) .  This  symbol  is  also  a  fixed  size  and  marks 
only  the  extended  centerline  and  glidepath  inter¬ 
cept  point  (GPIP)  threshold.  The  threshold  mark¬ 
er  is  sized  to  equal  to  a  runway  width  of  175 
feet . 

All  of  the  HUD  formats  are  identical  with  the  exception  of 
the  different  symbols.  The  dynamics  and  techniques  associated 
with  the  framing  of  the  runway  symbols  are  identical. 

All  of  the  runway  symbols  are  referenced  to  the  glidepath 
intercept  point  (GPIP) .  For  instance,  with  a  perfect  overlay  us¬ 
ing  Symbol  A,  the  symbol  will  subtend  the  runway  beginning  at  the 
glideslope  intercept  point  and  extending  to  the  departure-end 
threshold.  The  approach-end  runway  threshold  is  not  included  in 
the  overlay. 

Three  types  of  intentional  errors  were  introduced  in  this 
experiment.  These  were  lateral  offsets,  longitudinal  offsets, 
and  directional  deviations.  These  are  illustrated  using  Figure  7. 
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■■xporj  merit  Overview 


The  primary  experiment  matrix  consisted  of: 


Power  approach  aircraft  configurations. 


Constant  motion  and  visual  path  temporal  distor¬ 
tions  180  msec  time  delay  in  the  pitch  motion  and 
64  msec  time  delay  from  motion  to  head-up  display 
(visual)  pitch  response). 


Constant  velocity  vector  dynamics. 


Three  contact  analog-type  runway  symbols. 


Variations  in  the  accuracy  by  which  the  projected 
runway  marker  overlaid  the  real-world  runway. 
The  deviations  were  in  the  longitudinal  (parallel 
to  centerline),  lateral  (perpendicular  to  runway 
centerline),  and  directional  (angular  deviations 
about  the  glidepath  intercept  point)  directions. 


HORIZON  LINE 

PITCH  LADDER  (1:1  PITCH  LADDER) 

HEADING  SCALE 
INDICATED  AIRSPEED 
BAROMETRIC  ALTITUDE 

FLIGHT  PATH  MARKER  (CAGED  IN  AZIMUTH  AT 
PILOT  OPTION) 

WATERLINE  (PITCH  MARKER) 

COMMAND  BAR  FOR  HUD  TRACKING  TASK 
(IF  SELECTED) 


Figure  1 

Baseline  Display (Generic  HUD  Format) 


POTENTIAL  FLIGHT 
PATH  MARKERS  N 


VI 


FLIGHT  PATH 
MARKER 

•  Indicating  Thrust  >  Drag  or  Increasing 
Rate  of  Change  of  Flight  Path 


Figure  5 

Potential  Flight  Path  Symbol 


Low  AOA  On  AOA  High  AOA 

(  fast  )  (,¥s lowH) 


Figure  3 

Angle  of  Attack  Symbol 
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+1.2  DME 

1.  HORIZON  LINE 

2.  PITCH  LADDER 

3.  6LIDESL0PE  ERROR 

4.  HEADING  SCALE 

5.  INDICATED  AIRSPEED 

6.  BAROMETRIC  ALTITUDE 

7.  AIRMASS  FLIGHT  PATH  MARKER  (CAGED  IN  AZIMUTH  AT  PILOT  OPTION) 

8.  WATERLINE  (PITCH  MARKER) 

9.  RUNWAY  SYMBOL 

10.  MODE  INDICATOR 

11.  DISTANCE,  PARALLEL  TO  CENTERLINE,  TO  RUNWAY  GP1P  (NM) 


Figure  4 


Runway  Symbol  A  (Full  Runway  Outline) 
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1.  HORIZON  LINE 

2.  PITCH  LADDER 

3.  GLIDESLOPE  ERROR 

4.  HEADING  SCALE 

5.  INDICATED  AIRSPEED 

6.  BAROMETRIC  ALTITUDE 

7.  AIRMASS  FLIGHT  PATH  MARKER  (CAGED  IN  AZIMUTH  AT  PILOT  OPTION) 

8.  WATERLINE  (PITCH  MARKER) 

9.  RUNWAY  SYMBOL  B  (Dimensions  2000'  x  500') 


Figure  5 

Runway  Symbol  B  (Sidelines  Only) 
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HORIZON  LINE 
PITCH  LADDER 
CLIDESLOPE  ERROR 
HEADING  SCALE 
INDICATED  AIRSPEED 
BAROMETRIC  ALTITUDE 

AIRMASS  FLIGHT  PATH  MARKER  (CAGED  IN  AZIMUTH  AT  PILOT  OPTION) 
WATERLINE  (PITCH  MARKER) 

RUNWAY  SYMBOL  (GPIP  MARKER  175'  IN  WIDTH) 


Figure  6 

Runway  Symbol  C  (Centerline  Only) 
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LONGITUDINAL  DISPLACEMENT 


Figure  7 

Intentional  Errors  Introduced 


IV.  CONDUCT  OF  THE  EXPERIMENT 


This  experiment  was  performed  using  the  USAF  variable  sta¬ 
bility  NT-33A  aircraft.  The  evaluation  procedures  were  tailored 
to  the  maximum  extent  possible  to  satisfy  the  objectives  of  the 
exper i men t . 


A.  Evaluation  Pilots 


Five  evaluation  pilots  were  used  in  the  program.  The  experi¬ 
ment  matrix  was  sized  and  planned  for  four  evaluation  pilots. 
The  last  piloting  billet  was  thus  shared  by  two  pilots.  Because 
of  scheduling  problems,  the  final  evaluation  totals  were  not 
evenly  split  among  the  four  piloting  slots. 


B.  Evaluation  Procedures 


Each  evaluation  pilot  was  briefed  as  to  the  objectives  of 
the  experiment  and  the  head-up  display  formats  that  they  would  be 
flying.  Particular  attention  was  made  to  defining  the  display 
formats  and  features  because  some  of  these  were  new  to  the  pilot 
or  contradictory  in  sense  or  meaning  to  what  they  were  accus¬ 
tomed.  Informal  comments  and  discussion  with  the  evaluation  pi¬ 
lots  were  solicited  to  examine  their  initial  thoughts  and  reac¬ 
tions.  They  were  later  re-questioned  after  some  flight  experi¬ 
ence  regarding  the  displays  and  formats. 

Informal  pilot  comments  were  also  made  during  the  course  of 
the  evaluation.  Also,  the  pilots  were  asked  to  respond  to  a  pi¬ 
lot  comment  card  at  the  end  of  an  evaluation  (Table  II).  The  in¬ 
tent  of  the  comment  card  was  to  provide  a  common  question-re¬ 
sponse  series  for  each  of  the  pilots  and  configurations.  The  pi¬ 
lot  rating  was  made  with  reference  to  the  Cooper-Harper  pilot 
rating  (CHPR)  comment  card  (Figure  8,  from  reference  14) . 

Following  the  up-and-away  evaluation  which  lasted  15  minutes 
on  the  average,  the  power  approach  evaluations  were  flown  to  ei¬ 
ther  Niagara  Falls  Airport  (Runway  28R)  or  Greater  Buffalo  Inter¬ 
national  Airport  (Runway  5  and  23). 


Evaluation  Tasks 

For  power  approach  configurations,  the  evaluation  consisted 


The  evaluation  pilot  (EP) ,  flying  on  instruments 
(blue  visor  lowered),  intercepted  the  ILS  local- 


•  The  evaluation  pilot  then  flew  the  approach  to 
decision  height  using  the  HUD  as  the  primary 
flight  reference. 

•  At  decision  height,  the  EP  raised  the  blue  visor 
and  transitioned  to  a  visual  lineup  and  flare.  A 
20  ft  AGL  low  approach  was  executed. 

•  Following  the  approach,  aircraft  control  was 
passed  to  the  safety  pilot  while  the  EP  completed 
the  pilot  comments  and  CHPR. 

•  This  sequence  was  repeated  again  with  the  EP  per¬ 
forming  a  touch  and  go  landing  after  breakout  to 
visual  . 

•  A  third  approach  to  landing  could  be  performed  at 
the  discretion  of  the  EP. 

The  decision  height  was  varied  as  part  of  the  experiment 
from  a  maximum  of  200  ft  AGL  to  40  ft  AGL.  Typically,  a  100  ft 
decision  height  was  used. 

Each  approach  was  dictated  as  a  must  land  situation.  The 
evaluation  pilots  did  not  have  any  prior  knowledge  of  the  simu¬ 
lated  configuration  characteristics. 

Integral  to  the  Cooper-Harper  rating  scale  is  the  definition 
of  the  required  aircraft  task  and  task  performance  standards.  For 
the  up-and-away  evaluation,  the  aircraft's  required  task  included 
all  of  the  maneuvers  and,  hence,  the  rating  for  the  configuration 
was  based  on  performance  in  all  maneuvers.  The  task  performance 
standards  were  as  defined  in  Table  III. 


Table  I 


EVALUATION  PILOTS 


Pilot 

Flying 

Aircraft 

HUDs 

Flown 

Ident 

Time 

Background 

(a) 

A 

2700 

Air-to-Air 

F-15 

B 

2700 

Transport 

C 

2900 

Air-to-Ground 

F-16 

D 

14000 

Transport 

(b) 

E 

3600 

Reconnaissance, 

A- 7  , 

A-10, 

Flight  Test 

F-15, 

F-16 

(a) 

All  evaluation 

pilots  had  flown  HUD 

evaluations 

in  simulators 

of  various  types. 

(b) 

Pilot  D  had  flown  several  HUD  evaluations 

in 

flight  . 

Table  II 

PILOT  COMMENT  CARD 


•  Assign  overall  Cooper-Harper  Pilot  Ratings 

•  (Describe)  effect  of  aircraft  handling  qualities 
on  task  performance  and  pilot  workload: 

•  Up-and-away 

-  simulated  air-to-air 

-  air-to-ground 

-  acroba tics/unusual  attitude  recovery 

•  Powered  Approach 

-  approach 

-  flare  and  landing/waveof f 

•  Was  the  display  (overall)  adequate  for  mission? 

•  Effect  of  display  on  task  performance 

•  Effect  of  display  on  pilot  workload 

•  Good  feature(s)  of  display: 

•  Bad  feature(s)  of  display: 

•  Were  display  problems/deficiencies  a  function  of 

-  task 

-  flight  conditions  (VMC/IMC) 

•  Any  factor  in  evaluation  due  to 

-  turbulence? 

-  others? 

•  Summa ry/overal 1  comments 

-  any  change  in  rating 


Table  III 


EVALUATION  TASK  PERFORMANCE  STANDARDS 
(POWER  APPROACH) 


Desired  Performance  Standards  Adequate  Performance  Standards 


ILS  Approach 

No  P 10  Glide  slope  and  localizer  er- 

G1 ide  slope  and  localizer  er-  rors  less  than  1  deg  for  task, 

rors  less  than  1/3  deg  50  %  of 
task,  less  than  2/3  deg  remain¬ 
der  of  task. 

Visual  Landing 

No  D10 

iouchdown  within  5  ft  of  center 
line  and  within  250  ft  of  aim 
point  . 


Touchdown  within  5  ft  of  cen¬ 
terline  and  within  250  ft  of 
aim  point. 


-24- 


HANDLING  QUALITIES  RATING  SCALE 


ADEQUACY  f  OR  » ELECTED  TASK  OH 
REQUIRED  OPERATION* 


AIRCRAFT 

CHARACTERISTIC! 

DEMANDS  ON  THE  PILOT 
m  SELECTED  TASK  OR  Rf  OUfRCD  OPERATION* 

Excellent 

Highly  desirable 

Pilot  compensation  not  a  (actor  tor 
deaired  performance 

Good 

Negligible  deficiencies 

Pilot  compensation  not  a  factor  tor 
deaired  performance 

Fair  —  Some  mildly 
unpleasant  deficiencies 

Minimal  pilot  compensation  required  tor 
deaired  performance 

^  la  it  > 

aatiafactory  without 
improvement  v 


Deficiencies 

-  warrant 
1  Improvement 


Minor  but  annoying 

Deaired  performance  requires  moderate 

deficiencies 

pilot  compensation 

Moderately  objectionable 

Adequate  performance  require* 

deficiencies 

considerable  pilot  compensation 

Very  ob|tctionable  but 

Adequate  performance  requires  extensive 

tolerable  deficiencies 

pilot  compensation 

/  »»  adequate  X. 
/  parlor  manca  \ 

^attainable  with  a  lovable, 
X.  pilot  workload7  S 


\  Deficient!** 
*  require 
I  improvement 


Maior  deficiencies 

Adequate  performance  not  attainable  with 
maximum  tolerable  pilot  compensation 
CenirollabUt ty  not  in  goes  fron 

Major  deficiencies 

Considerable  pilot  compensation  ta  required 
for  Con*  rot 

Major  dificisnciet 

Intense  pilot  compensation  la  required  to 
retain  control 

H  controllable1 


1  Improvement 
P  mandatory  | 


M«| or  deficiencies 


Control  will  be  lost  during  tome  portion  of 
required  operation 


Cooper  M*»P#|  n*l  NASA  TNO  SIM 


•  p«tir.i*on  ot  reowrae  operauon  M«pt«aa  eewgnstiqn  of  tiiphi  pnaM  ond  <\ 
awt>pn«M*  wrttl  Ktomptnyrtig  conAioni 


Figure  8 

Pilot  Rating  Scheme  (from  Reference  14) 


V.  EXPERIMENT  RESULTS 


The  results  of  this  in-flight  experiment  are  presented  in 
hi?  sect- ion  with  discussion  and  additional  data  correlations. 


A  Flight  Program  Summary 

This  flight  program  was  performed  in  two  phases.  Phase  I 
was  flown  in  October  1985  and  Phase  II  in  January  1986.  All  fly¬ 
ing  originated  from  the  Calspan  Flight  Research  Facility  in  Buf¬ 
falo,  New  York.  The  program  consisted  of  36  flights  totaling 
49.5  flying  hours.  34  piloted  evaluations  were  performed  as  part 
of  this  study.  All  of  the  evaluation  pilots  participated  in  the 
rwo  flight  phases  with  the  exception  of  Pilot  C  who  became  un¬ 
available  for  Phase  II  at  the  last  moment  due  to  a  schedule  con¬ 
flict.  The  breakdown  of  flights  and  evaluations  by  pilot  is 
shown  in  Table  IV  below. 

Each  of  the  pilots  was  given  an  orientation/practice  evalua¬ 
tion  to  become  familiar  with  the  evaluation  tasks  and  procedures. 


B .  Experimental  Data 

The  experiment  data  consist  of  pilot  rating,  pilot  comments, 
and  task  performance  records.  The  task  performance  records  in¬ 
clude  data  recorded  on  an  on-board  AR-700  digital  flight  recorder 
and  video  taken  by  a  camera  mounted  just  aft  of  the  HUD  combining 
glass.  The  pilot  comments  for  each  evaluation  are  available  in 
Reference  (8) . 

Table  V  and  Appendix  I  summarize  the  results. 

1 .  Effect  of  Display  Inaccuracies  on  Pilot  Corrections 

Intentional  offsets  of  up  to  1500  ft  longitudinal  and  200  ft 
laterally  were  flown.  Heading  deviations  were  scheduled  for 
evaluation  but  were  not  flown  because  of  time  and  funding  con- 

s t  ra  i  n  t  s . 

On  each  approach,  the  pilots  were  asked  to  place  an  event 
marker  on  the  data  at  breakout.  The  event  marker  was  activated 
by  squeezing  the  trigger  of  the  front  cockpit  centerstick.  This 
placed  a  discrete  voltage  signal  onto  the  digital  onboard  record¬ 
er  and  an  "X"  was  drawn  on  the  HUD.  The  event  marker  was  used  to 
investigate  if  any  control  mi s- j udgmen t s  were  made  in  the  face  of 
runway  misalignments.  A  typical  time  history  is  shown  in  Figure 


w 


Examination  of  the  approach  time  histories  do  not  show  any 
initial  incorrect  control  inputs  (initial  control  inputs  in  wrong 
direction).  The  pilot  comments  did  not  indicate  any  problems  in 
picking  up  the  real  runway  or  transitioning  to  outside  cues  from 
the  head-up  display.  One  exception  was  on  Pilot  A's  first  flight 
(150  ft  lateral  offset,  symbol  C)  where  he  noted  difficulty  in 
finding  the  real  runway  after  breakout. 

The  various  runways  apparently  did  not  dramatically  influ¬ 
ence  the  pilots  transitioning  to  outside  visual  flight  referenc¬ 
es.  Aside  from  the  single  instance  noted  above,  the  pilots  had 
no  problems  in  the  HUD- to-visua 1  transition.  HUD  fixation  was 
not  reported  to  be  a  problem. 

By  virtue  of  the  display  design,  intentional  longitudinal 
errors  short  of  the  glidepath  intercept  point  (GPIP)  were  more 
tolerable  than  errors  past  the  GPIP.  All  runway  symbols  were 
reference  to  and  drawn  from  the  GPIP.  The  full  runway  display 
actually  excluded  the  distance  from  the  runway  threshold  to  the 
glidepath  intercept  point;  thus,  an  intentional  error  of  up  to 
approx ima tel y  1000'  short  of  the  GPIP  would  still  draw  the  runway 
projection  onto  the  actual  runway  surface.  For  longitudinal  de¬ 
viations  past  the  GPIP,  the  primary  difficulty  facing  the  pilot 
at  breakout  is  landing  long  or  the  requirement  to  go  around. 

2 •  Inter-Pilot  Rating  Comparison 

An  inter-pilot  rating  comparison  cannot  be  made  because  of 
insufficient  overlap  between  pilots  in  this  task.  Also,  an  ap¬ 
parent  dilemma  in  the  assignment  of  a  CHPR  in  the  face  of  inten¬ 
tional  runway  display  offsets  occurred.  This  dilemma  involved 
questions  such  as: 

•  Was  the  display  acceptable  if  it  provided  guid¬ 
ance  to  some  incorrect/erroneous  location? 


•  Did  the  display  deficiencies  warrant  or  require 
improvements? 

•  If  it  was  possible  to  land  the  aircraft  with  de¬ 
sired  performance  despite  some  display  offset  at 
breakout,  was  the  display  deficient  or  did  it 
need  improvement? 

Each  of  these  questions  was  approached  differently  by  each 
pilot  and  sometimes  the  pilots  applied  different  standards  on 
different  approaches.  The  ratings  were  not  consistent.  The  pi¬ 
lot  comments  are  invaluable,  however.  Also,  the  ratings  and  com¬ 
ments  contained  personal,  although  unquantified,  biases  in  terms 
of  which  synthetic  runway  they  preferred. 


Effect  of  Display  Format. 


3  . 


Thioe  contact,  analog  runway  displays  were  mechanized.  These 
varied  significantly  in  terms  of  attractiveness  and  detail.  This 
program  represented  the  first  exposure  to  a  contact  analog  runway 
format  for  three  of  the  pilots  (Pilots  A,  B,  and  C) .  Few  prob¬ 
lems  were  noted  by  the  pilots  in  adapting  t.o  the  display  format 
with  the  one  exception  being  initial  complaints  of  excessive  pi¬ 
lot  workload. 


The  pilots  in  their  initial  flights  gave  some  critique  to 
the  landing  display  format  in  general.  These  initial  thoughts 
are  presented  for  completeness  in  terms  of  a  lessons  learned  sec¬ 
tion  defining  the  HUD  format  deficiencies.  These  deficiencies 
were  not  remedied  for  this  program  nor  do  they  significantly  af¬ 
fect  the  results.  (Comments  regarding  the  air  mass  flight  path 
for  the  power  approach  task  have  already  been  presented  and  are 
not  repeated  here).  The  potential  display  deficiencies  were: 

•  The  display  lacked  a  bank  angle  index.  Four  pi¬ 
lots  felt  that  a  bank  index  was  required  for  the 
power  approach  task  (None  felt  it  was  required 
for  the  up-and-away  flight  tasks). 

•  The  display  lacked  rate  information.  Pilot  D 
continually  bemoaned  the  absence  of  explicit  rate 
parameters  such  as  airspeed  rate  or  altitude 
rate. 

•  Angle  of  attack  bracket  was  drawn  with  respect  to 
the  waterline  and  to  the  velocity  vector.  When 
the  velocity  vector  was  uncaged  in  azimuth,  the 
angle  of  attack  bracket  and  velocity  vector  were 
not  necessarily  beside  each  other.  This  was  an 
annoyance  because  both  symbols  were  treated  as 
primary  control  parameters.  Any  significant  dis¬ 
placement  between  the  two  symbols  increased  the 
pilot's  scan  pattern  and  attendant  pilot  work- 
1  oad . 

Four  pilots  clearly  preferred  Symbol  A  (the  full  runway  dis¬ 
play).  This  runway  display  was  felt  to  provide  the  best  percep¬ 
tual  cues  in  the  approach. 

Symbol  B  (sidelines  only  display)  was  not  well  received  pri¬ 
marily  because  of  its  short  runway  sidelines  (2000  ft)  and  width 
(500  ft).  The  short  runway  length  was  felt  to  limit  the  perspec¬ 
tive  cues  for  line-up  that  were  inherent  to  the  other  symbols. 
For  the  2nd  phase  of  evaluations,  the  width  of  Symbol  B  (Symbol 
3'  )  was  reduced  to  250'  and  the  symbol's  acceptance  was  im¬ 

proved.  One  good  feature  of  Symbol  B  was  found  to  be  that  the 
runway  sidelines  formed  a  natural  box  for  which  to  fly  and  keep 
the  flight  path  inside  of  on  the  approach.  This  was  felt  to  be  a 
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nice  feature.  Deviations  of  the  flight  path  outside  the  box  were 
easily  noted,  thus  providing  a  strong  cue  feedback  to  the  pilot. 

Symbol  C  was  sometimes  preferred  over  Symbol  B  as  the  second 
choice  behind  Symbol  A.  The  approach  course  line  up  was  felt  to 
be  strong  for  this  symbol,  as  one  might  expect  from  its  charac¬ 
teristics.  Also,  glidepath  deviations  could  be  easily  picked  out 
because  of  the  spareness  of  the  display. 

The  results  of  this  program  are  certainly  valid  with  two 
caveats.  First,  intentional  runway  obstructions  were  not  at¬ 
tempted  to  test  whether  the  pilot's  were  actually  experiencing 
HUD  fixation.  It  has  been  previously  reported  that  in  ground- 
based  simulations,  pilots  were  unaware  of  runway  obstructions 
during  flight  with  a  HUD-equipped  a i rcra f t  ( 1_5 )  .  For  instance, 
pilots  did  not  see  aircraft  taxiing  onto  the  active  runway  during 
an  approach  to  landing.  It  is  doubtful  that  the  pilots  would 
have  had  any  problems  seeing  obstructions  in  this  experiment. 
The  visual  cues  in  this  program  are  real  and  the  visibility  con¬ 
ditions  were  typically  excellent  unlike  what  may  be  presented 
during  ground-based  simulation  experiments.  Also,  the  blue/amber 
IMC  simulation  may  influence  this  result.  In  operation,  the  run¬ 
way  displays  and  any  symbol  inaccuracy  would  become  critically 
important  in  an  actual  IMC  situation.  In  the  blue/amber  simula¬ 
tion,  IMC  flight  is  conducted  up  to  a  VMC/full  daylight  breakout. 
No  shades  of  grey  or  low  visibility  conditions  were  simulated. 
It  may  be  the  case  that  HUD  fixation  during  the  transition  from 
inside  to  outside  visual  cues  and  the  identification  of  runway 
display  errors  are  hindered  more  under  low  visibility  conditions. 

4 .  Effect  of  Crosswinds 

Four  evaluations  were  flown  for  this  task  in  high  crosswind 
environments  and,  in  each  case,  poor  ratings  were  given  because 
of  the  f iel d-of -v iew  limitations.  The  rating  degradation  was  1 
to  2  CHPRs .  In  the  presence  of  a  crosswind,  the  contact  analog 
i unway  display  will  be  offset  from  the  center  of  the  HUD.  The 
NT- 3 3  HUD  has  an  approximately  16  deg  instantaneous  lateral  FOV 
or  +/-8  deg  from  center.  For  crab  angles  of  greater  than  7  deg, 
the  runway  display  symbology  can  be  blanked  because  of  FOV  limi¬ 
tations.  When  this  occurred,  the  evaluation  pilots  had  to  tran¬ 
sition  to  alternative  landing  guidance  in  the  approach.  Unfortu¬ 
nately,  for  large  crab  angles,  no  landing  guidance  at  all  is  pro¬ 
vided  in  the  HUD.  This  is  a  limitation  of  the  current  DEFT  de¬ 
sign  and  it  was  not  altered  for  this  program.  This  issue  and  its 
solution  were  not  a  part  of  the  experiment.  These  crosswind 
evaluations  are  excluded  from  the  analytic  discussion  since  FOV 
was  not  a  specific  experiment  variable. 

5 .  Interaction  with  Display  Dynamics  Experiment 

The  pilot  comments,  as  delay  was  added  to  the  display,  note 
increased  and  possibly  annoying  "bouncing"  of  the  display.  The 
experimentally  added  delay  was  such  that  the  entire  display  was 


uniformly  delayed.  Of  particular  concern  was  the  movement  of  the 
conformal  runway  which  provided  the  landing  guidance  information. 
The  pilot  rating  and  flying  qualities  evaluation  were  decided  by 
whether  the  pilot  could  compensate  for  the  bouncing  of  the  dis¬ 
play  or  whether  the  bouncing  deficiency  warranted  or  required  im¬ 
provement.  The  pilot  compensation  for  the  bouncing  was  primarily 
to  estimate  the  mean  position  of  HUD  symbols  through  their  range 
of  movement  and  attempt  to  control  that  movement.  It  was  not  the 
case  that  control labil i ty  problems  occurred.  At  the  highest  time 
delay  values,  the  pilots  were  seriously  questioning  the  validity 
of  the  display  and  knowing  that  the  movement  on  the  display  was 
not  caused  by  turbulence  or  control  inputs.  In  this  case,  the 
pilots  ceased  to  track  the  velocity  vector  and  runway  display 
t ■ ghtly. 

6 .  Additional  Observations 

In  these  evaluations,  a  contact  analog  runway  symbology  was 
used  as  the  baseline  HUD  display.  For  the  majority  of  the  evalu¬ 
ation  pilots,  it  was  their  first  exposure  to  this  type  of  dis¬ 
play.  in  general,  the  runway  display  symbology  was  found  to  be 
ar.  effective,  safe,  and  natural  method  for  landing  guidance.  The 
pilot  comments  were  virtually  unanimous  in  indicating  acceptance 
of  the  display.  From  informal  comments,  the  only  major  deficien¬ 
cy  of  the  display  format  was  the  air  mass  velocity  vector.  The 
majority  of  pilots  expressed  a  desire  for  an  inertial  velocity 
vector.  This  desire  may  have  also  been  prompted  by  the  lack  of  a 
t rack  angle  marker  for  the  air  mass  velocity  vector.  The  iner¬ 
tial  velocity  vector  may  not  have  been  necessary  if  a  track  mark- 
e r  was  p r ov i d ed . 

The  conformal  runway  display  (even  without  any  added  display 
delay)  was  sometimes  thought  to  be  unrealistic  in  its  movement, 
particularly  in  response  to  control  inputs  or  turbulence.  The 
conformal  display  was  said  to  bounce  excessively  and  be  overly 
sensitive.  In  actuality,  the  movements  of  the  runway  display 
were  not  excessive  nor  inaccurate.  The  displayed  runway  position 
and  its  relative  motion  were  an  accurate  representation  of  the 
outside  world.  The  pilot's  observations  were  of  a  perceptual 
phenomenon.  The  runway  outline  display  is  projected  against  a 
virtual  black  (IMC)  background.  This  display  provides  the  pilot 
a  relatively  small  foveal  viewport  to  the  outside  world.  Thus 
control  movements  or  turbulence  which  cause  aircraft  attitude 
viango-  create  large  d i spl acements  relative  to  the  total  FOV  of 
the  HUD.  This  perception  was  noted  primarily  on  the  first 
i rights  and  subsided  as  experience  grew.  Certainly  learning  ef¬ 
fects  played  a  role  in  adapting  to  this  display. 

‘lire  effect  of  such  a  sma  1 1  FOV  on  approach  to  landing  using 
a  periscope  showed  a  tendency  for  pilots  to  mis-judge  the  ap¬ 
proach  and  fly  consistently  low(l_6).  No  such  effect  was  noted 
i n  t  h is  study. 


The  different  reference  frames  between  the  inertial  and  the 
air  mass  velocity  vector  were  a  significant  factor.  In  numerous 
instances,  the  nominal  air  mass  velocity  vector  was  criticized 
because  of  its  inability  to  depict  precisely  the  flight  path  in 
relation  to  the  HUD  runway  display.  The  pilots  did  not  feel  that 
they  were  getting  the  help  from  the  display  that  was  possible 
because  of  the  need  to  correct  explicitly  for  wind  variations. 
This  situation  was  particularly  acute  for  crosswind  situations, 
where  it  was  necessary  to  hold  the  velocity  vector,  some  lateral 
distance  from  the  "no-wind"  aimpoint  on  the  runway  display.  This 
situation  is  alleviated  totally  with  an  inertial  velocity  vector 
and  might  have  been  eased  by  the  addition  of  a  track  marker  to 
the  air  mass  referenced  display. 


Table  IV 

EVALUATION  SUMMARY 


Pilot 
Identi  - 
f ica  t ion 

FI i ghts 

Evaluations 

Task  A  (a) 

This  Study 

Total 

A 

10 

38 

13 

51 

B 

7 

26 

5 

31 

C 

4 

19 

6 

25 

D 

4 

5 

8 

13 

E 

3 

8 

2 

10 

Total 

28 

96 

34 

130 

(a)  Dynamic  Response  Study  [8) 
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Table  V 

SYMBOL  ACCURACY  EVALUATION  SUMMARY  (a) 


Of  f  3 

et 

Symbol  A 

Symbol  B  Symbol  C 
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3  . 1 
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4.5 

11 
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ft 

1 

6.0 
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3 
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ft 

1 
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8 
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Lung 

r\  f  f 

.  -500 
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1 
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1 

4.0 
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2 
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2 
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ft 

2 

2.5 

1 

4.0  1 

5.0 

4 

3.5 

[  D  } 

+  1000 

ft 

2 

5.0 

3 

4.2 

5 

4.5 

Summa ry 

15 

3.9 

10 

4.4  9 

5.8 

34 

4.3 

(a ) 

(b) 

Key: 

Minus 

Number 

denotes 

of  evaluations  Average  Rating 

short  of  GPIP;  Plus  denotes  past 
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Figure  9 

Approach  and  Landing  Time  History  Example 
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VI.  CONCLUSIONS 


An  in-flight  investigation  of  the  effect  cf  head-up  display 
sytbol  accuracy  requirements  was  performed  using  the  USAF  NT-3  3A 
aircraft.  Up  to  200  ft  lateral  offsets  and  1500  ft  longitudinal 
deviations  were  flown.  The  results  of  this  study  suggest  that: 

•  These  deviations  were  clearly  evident  at  visual 
breakout  and  at  least  adequate  landing  perfor¬ 
mance  was  attainable. 

»  Control  activity  at  breakout  did  not  indicate  any 
problems  or  errors  in  judgment  in  the  transition 
from  IMC  to  VMC.  Only  one  instance  was  there  a 
problem  noted  in  visually  acquiring  the  real  run¬ 
way.  This  instance  was  the  first  approach  by  one 
evaluation  pilot. 

•  The  runway  perspective  symbols  was  easily  flown 
by  the  evaluation  pilots. 

•  The  evaluation  pilots  preferred  the  complete  run¬ 
way  outline  with  the  centerl ine/threshold  symbol 
rated  second. 

Several  experimental  factors  may  have  influenced  these  re¬ 
sults.  Among  these  are  the  unrealistic  transition  from  total  in¬ 
strument  conditions  to  total  visual  conditions  with  good  visibil¬ 
ity.  The  effect  of  visual  conditions  with  very  limited  visibili¬ 
ties  was  not  simulated. 

The  conclusions  do  not  support  the  MD-80  experience  which 
indicate  a  highly  realistic  runway  symbol  is  overly  compelling 
and  can  cause  a  pilot  to  ignore  real  world  cues.  They  also  do 
not  support  the  observations  made  during  the  MARS  HUD  studies  in 
which  the  pilots  ignored  real  world  cues  and  flew  HUD  symbols. 

They  do  support  early  HUD  simulations  in  which  pilots  tended 
to  fly  real  world  cues  and  not.  erroneous  HUD  symbols. 

This  may  be  a  result  of  the  evaluation  pilots  distrust  of 
■ne  air  mass  velocity  vector  coupled  with  the  knowledge  that  the 
HUD  symbols  were  being  degraded. 

The  results  indicate  that  accurate  position  information  is 
rot.  a  requirement  for  a  HUD  to  be  useful  during  an  ILS  approach. 
Tha t  is,  an  inertial  navigation  system  is  not  required.  However, 
based  on  earlier  experience,  accurate  pitch,  roll,  and  heading 
la "a  is  essential  for  a  usable  HUD.  For  this  reason,  a  inertial 
quality  gyro  platform  (AHRS)  will  probably  be  required. 


It  does  appear  that  the  use  of  Cooper-Harper  Pilot  Ratings 
(CHPR)  for  display  evaluations  does  not  supply  enough  data  to 
adequately  rate  and  rank  the  various  symbol ogies .  Future  flight 
data  cards  require  display  specific  questions.  The  CHPRs  alone 
can  be  masked  by  handling  issues.  In  the  present  study,  CHPRs 
were  necessary  to  deal  with  the  display  dynamics  portion  of  the 
flight.  In  this  study,  unf  ortuna  tely,  we  elected  not  to  ask 
enough  of  the  right  questions  concerning  the  effect  of  the  vari¬ 
ous  formats. 

By  their  nature,  CHPRs  are  mission-related.  In  display  sym¬ 
bology  research,  the  display  may  be  intentionally  inadequate  for 
the  mission  (as  in  this  study) .  This  virtually  guarantees  rat¬ 
ings  of  4,  5,  and  6. 


VII.  RECOMMENDATIONS 


following  recommendations  can  be  made: 

The  use  of  an  inertial  navigation  system  is  not 
required  for  future  HUDs.  Other  studies  do  indi¬ 
cate  that  inertial  quality  gyroscopic  platforms 
will  be  required  for  proper  attitude  sensing 
(particularly  for  a  heading  reference). 

A  perspective  runway  symbol  is  more  than  adequate 
for  ILS  approaches. 

A  full  perspective  runway  symbol  (Symbol  A)  is 
preferred.  Based  on  previous  HUD  experience  in 
poor  visibility,  the  use  of  a  less  compelling 
display  (Symbol  B)  is  recommended  for  airplanes 
not  equipped  with  inertial  navigation  systems. 

Future  studies  of  this  type  should  develop  means 
to  simulate  restricted  visibility.  One  approach 
might  be  to  have  the  evaluation  pilot  wear  two 
thicknesses  of  blue  and  remove  only  one.  If  the 
intensity  of  the  blue  were  adjusted  properly,  a 
faint  view  of  the  external  world  would  be  avail¬ 
able. 

Future  studies  evaluating  variations  of  a  base¬ 
line  display  should  use  display  specific  ques¬ 
tionnaires  designed  to  point  out  the  differences 
being  examined.  Handling  qualities  ratings  may 
not  yield  sufficient  information. 
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APPENDIX  I 


SYMBOL  ACCURACY  EVALUATION  RESULTS  (a) 
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Key: 

Pilot : 

Pilot 

Ra  t  i  ng 

■  (Configuration) 

Below  line  symbols  were  B' 

Minus  denotes  short  of  GPIP;  Plus  denotes  past  GPIP 
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